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DETAILED ACTION 
Response to Amendment 

The amendment submitted January 9, 2008 lias been entered. 

The objections to the drawings, specification and claim 1 1 have been withdrawn. 

Response to Arguments 
Applicant's arguments with respect to claimi , 10 and 1 1 have been considered 
but are moot in view of the new ground(s) of rejection. 

Claim Objections 

Claim 1 is objected to because of the following informalities: In the 3'"'^ line of the 
paragraph starting, "Means for controlling movement", the examiner believes the phrase 
"recognized by said means for recognizing a distance to the subject, at a predetermined 
distance" should read "recognized by said means for recognizing a subject to be 
followed up, at a predetermined distance". Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

Claims 1-2 and 5-8 rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent No. 6,556,892 to Kuroki et al. (Kuroki) in view of "Vision Based Tracking 
with a Mobile Robot, Christian Schlegel, Jorg lllmann, Heiko Jaberg, Matthias Schuster, 
Robert Worz, 1998, (Schlegel). 
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Regarding amended independent claim 1, "An apparatus for controlling a 
movable robot comprising a camera, moving means, and a device for outputting a 
sound, which comprises: "- reads on Kuroki Fig 4, where a movable robot, having a 
picture input device (251 ), left and right legs, (331 R and 331 L) and a speech output 
device (253) is shown - Fig. 4 is described at col. 6, line 3 to col. 7 line 36. 

"means for recognizing a subject to be followed up, which recognizes the subject 
on the basis of an image taken by the camera;" - reads on the authentication processor 
(114) of Kuroki Fig. 5, described at Col. 8 lines 44-56 which uses previously stored face 
images, and images of other body parts to authenticate (recognize) a valid user of the 
system. Even if Kuroki's authentication processor was insufficient for recognizing a 
subject to be followed up, Schlegel on page 420, paragraph 3.1 .1 states that at startup, 
the person to be followed introduces itself to the robot. It would have been obvious to 
one of ordinary skill in the art at the time of the invention to augment the prior art 
method of Kuroki with the Schlegel's introduction to yield a positive identification of the 
subject to be followed. 

"means for recognizing a distance to from the subject having been recognized by 
the means for recognizing a subject to be followed up;" - Kuroki does not measure the 
distance to the subject to be followed up, but Schlegel, which uses dual cameras 
instead of the single camera of Kuroki, determines the distance between the robot and 
the person using the disparity between the two simultaneous images as detailed on 
page 425, 1®' full paragraph. It would have been obvious to one of ordinary skill in the 
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art at the time of the invention to use the known technique of using dual cameras to 
improve Kuroki's robot by allowing it to measure distance to viewed objects. 

"means for controlling movement, which controls said moving means so as to 
keep the distance from said movable robot to the subject, having been recognized by 
said means for recognizing a distance to the subject, at a predetermined distance; and " 
- Kuroki does not follow the identified subject, but Schlegel follows the identified subject 
since it is a follower robot. Further, Schlegel maintains a distance between the robot 
and the person as mentioned on page 425, line 5-8. It would have been obvious to one 
of ordinary skill in the art at the time of the invention to apply Schlegel's known 
technique of following to the subject previously identified by Kuroki at a distance in order 
to have Kuroki's robot be more human-like. 

"means for controlling the outputting of a sound, which outputs a sound or a 
voice related to the distance to the subject" - Reads on Kuroki's speech synthesis unit 
Fig. 5 120 as described at col. 9, lines 16-22. While Kuroki does not detail the robot 
discussing the distance to the person, in col. 9, lines 9-12 it describes a dialog 
management unit 124 that generates a reply to a user input via a speech synthesis unit 
120 and subsequently outputs the reply via a loudspeaker. There is no reason to 
assume that the robot using the Kuroki/Schlegel combination could not be programmed 
to output a sound when the distance between the robot and the person became too 
large. 

wherein the sound or the voice from the means for controlling the outputting of a 
sound informs the subject about a situation regarding the distance to the subject, so as 
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to prompt the subject to keep the distance from said movable robot to the subject at the 
predetermined distance. Schlegel reports that its robot was able to track and follow 
persons robustly in real time; hence there was little reason for the tracking robot to 
speak. However, the capabilities in the Kuroki robot would have been able cause a 
Kuroki/Schlegel robot to utter words to inform the subject about the situation if needed. 

Regarding claim 2, which is dependent on Claim 1, "wherein said moving 
means of the movable robot moves by two legs' walking." - Reads on Kuroki's Legs 
(101 Rand 101 L) of Fig. 1 as described in col. 43-53. 

Regarding claim 5, which is dependent on Claim 1 , "wherein the subject to be 
followed up is a person, and which further comprises means forjudging instruction from 

a person." - - reads on Kuroki col. 7, lines 47-52 for person recognition and col. 8, lines 
24-34 for interpreting spoken commands such as "run", and "hurry". 

Regarding claim 6, which is dependent on Claim 5, "wherein said means for 
judging instruction from a person judges whether or not the robot follows up the person 
based on the results of recognition in which the person is recognized from the face 
image." - Reads on Kuroki col. 7, lines 47-52 for person authentication. Further reads 
on Schlegel where in paragraph 3.1.1 the person introduces itself and is recognized 
from the face and clothes being worn. This is helpful as when the person is being 
followed, they are no longer facing the robot. It would have been obvious to one of 
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ordinary skill in the art at the time of the invention to combine the authentication type of 
recognition of Kuroki and the situational identification of Schlegel to yield a more robust 
recognition. 

Regarding claim 7, which is dependent on Claim 6, "wherein said means for 
judging instruction from a person judges the instruction from the person based on at 
least one of posture, and gesture of said person." - - Reads on Kuroki col. 8, lines 52- 
54 for interpreting hand gestures. 

Regarding claim 8, which is dependent on Claim 5, "wherein said means for 
judging instruction from a person judges the instruction from the person based on a 
voice vocalized from said person." - reads on Kuroki col. 8, lines 24-34. 

Claims 3-4 are rejected under 35 U.S.C. 103(a) as being obvious over Kuroki in 
view of Schlegel and further in view of U.S. Patent Publication No. 2004/0230340 to 
Fukuchi et al. (Fukuchi). 

Regarding claim 3, which is dependent on Claim 1, "which further comprises 
means for holding map information, which holds map information of an area within 
which said movable robot moves, and" - While the robots of both Kuroki and Schlegel 
incorporate memory which could hold maps, maps are not mentioned. However, 
Fukuchi uses part of the memory to hold map information map of the local environment 
of the area - Fig. 5 and 1|19. It would have been obvious to one of ordinary skill in the 
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art at the time of the invention to incorporate the map holding mechanism of Fukuchi 
into the robot described by the combination of Kuroki and Schlegel to enhance the 
abstract input data on which Kurol<i models a behavior. 

"wherein said means for controlling movement determines the actuation of said 
moving means based on the map information held in said means for holding map 
information." - - While Kuroki and Schlegel are silent on maps, Fukuchi enhances its 
maps, determining landmarks, obstacles and safety zones. Fukuchi then uses these 
map features to guide movement - HI 9. Once the map was incorporated in the robot of 
Kuroki and Schlegel, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to use the data so contained. 

Regarding claim 4, which is dependent on Claim 3, "wherein a restricted area 
which prohibits approach is set in said map information held in said means for holding 
map information." Fukuchi describes recognizing mobility regions in its map -1|67 - and 
conversely does not enter regions that are not mobile areas - such as obstacles, 
landmarks and safety areas - 1|81 . It would have been obvious to one of ordinary skill in 
the art at the time of the invention to use these features of Fukuchi with the combined 
robot to better interact with the human environment. 



Claim 9 is rejected under 35 U.S.C. 103(a) as being obvious over Kuroki in view 
of Schlegel and further in view of UK Patent Application No. GB 2 258 098 to Na (Na). 



Application/Control Number: 10/814,782 Page 8 

Art Unit: 3661 

Regarding claim 9, which is dependent on Claim 1, "wherein said means for 
controlling the outputting of a sound changes a volume of voice outputted to said device 
for outputting a sound, based on a circumferential noise level." - Neither Kuroki nor 
Schlegel mentions changing the sound based on the ambient noise. However, Na 
teaches automatically controlling the volume of sound based on background noise. 
(Page 1 , lines 6-9). It would have been obvious to one of ordinary skill in the art at the 
time of the invention to incorporate the volume control feature of Na in the moving robot 
described by the combination of Kuroki and Schlegel to assure that the robot can be 
heard. 

Claims 10-11 are rejected under 35 U.S.C. 103(a) as being obvious over Kuroki 
in view of Schlegel and further in view of Japanese Patent No. JP-2005202078 A to 
Shimomura (Shimomura). Claims 10 and 11 are to a process for using the apparatus of 
claim 1 and to a computer readable medium containing a program to execute the 
process of claim 10. They will be treated together. 

Regarding amended independent claims 10 and 11, "A process for controlling a 
movable robot comprising a camera, moving means, and a device for outputting a 
sound, which comprises:" - - reads on Kuroki Fig 4, where a movable robot, having a 
picture input device (251), left and right legs, (331 R and 331 L) and a speech output 
device (253) is controlled - Fig. 4 is described at col. 6, line 3 to col. 7 line 36. 

"a step for recognizing a subject to be followed up, which recognizes the subject 
on the basis of an image taken by the camera;" - reads on the authentication processor 
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(114) of Kuroki Fig. 5, described at Col. 8 lines 44-56 which uses previously stored face 
images, and images of other body parts to authenticate (recognize) a valid user of the 
system. Even if Kuroki's authentication processor was insufficient for recognizing a 
subject to be followed up, Schlegel on page 420, paragraph 3.1 .1 states that at startup, 
the person to be followed introduces itself to the robot. It would have been obvious to 
one of ordinary skill in the art at the time of the invention to augment the prior art 
method of Kuroki with the Schlegel's introduction to yield a positive identification of the 
subject to be followed. 

"a step for recognizing a distance to the subject having been recognized by the 
step for recognizing a subject to be followed up;" — Kuroki does not measure the 
distance to the subject to be followed up, but Schlegel, which uses dual cameras 
instead of the single camera of Kuroki, determines the distance between the robot and 
the person using the disparity between the two simultaneous images as detailed on 
page 425, 1®' full paragraph. It would have been obvious to one of ordinary skill in the 
art at the time of the invention to use the known technique of using dual cameras to 
improve Kuroki's robot by allowing it to measure distance to viewed objects. 

"a step for controlling movement, which controls said moving means so as to 
keep the distance to the subject having been recognized by said step for recognizing a 
distance to the subject at a predetermined distance; and"- Kuroki does not follow the 
identified subject, but Schlegel follows the identified subject since it is a follower robot. 
Further, Schlegel maintains a distance between the robot and the person as mentioned 
on page 425, line 5-8. It would have been obvious to one of ordinary skill in the art at 



Application/Control Number: 10/814,782 Page 10 

Art Unit: 3661 

the time of tlie invention to apply Schlegel's known technique of following to the subject 
previously identified by Kuroki at a distance in order to have Kuroki's robot be more 
human-like. 

"a step for controlling the outputting of a sound, which outputs a sound or a voice 
related to the distance to the subject, " - - Reads on Kuroki's speech synthesis unit Fig. 
5 120 as described at col. 9, lines 16-22. While Kuroki does not detail the robot 
discussing the distance to the person, in col. 9, lines 9-12 it describes a dialog 
management unit 124 that generates a reply to a user input via a speech synthesis unit 
120 and subsequently outputs the reply via a loudspeaker. There is no reason to 
assume that the robot using the Kuroki/Schlegel combination could not be programmed 
to output a sound when the distance between the robot and the person became too 
large. Further while neither Kuroki nor Yoshinori mention changing the sound based on 
distance to the subject, Shimomura teaches changing the speech form, such as sound 
volume, speed and intonation based on the distance detected between the robot and 
the user - see Derwent 2005-53276 Basic Abstract. It would have been obvious to one 
of ordinary skill in the art at the time of the invention to incorporate the volume control 
feature of Shimomura in the moving robot described by the combination of Kuroki and 
Schlegel to further increase the human-like aspect of the robot. 

"wherein the sound or the voice from the step for controlling the outputting of a 
sound informs the subject about a situation regarding the distance to the subject, so as 
to prompt the subject to keep the distance from said movable robot to the subject at the 
predetermined distance." - Schlegel reports that its robot was able to track and follow 
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persons robustly in real time; hence there was little reason for the tracking robot to 
speak. However, the capabilities in the Kuroki robot would have been able cause a 
Kuroki/Schlegel robot to utter words to inform the subject about the situation if needed. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: Harai, N. and Mizoguchi, H. Visual Tracking of Human Back and 
Stioulder for Person-following Robot, Proceedings 2003 lEEE/ASME Int. Conf on 
Advanced intelligent Mechatronics (AIM 2003) pp. 527-532 for person following robot; 
Kaplan, F., Talking AIBO: First Experimentation of Verbal Interactions witli an 
Autonomous Four-legged Robot, Proceedings of the CELE-Twente workshop in 
interacting agents 2000 - for talking Robot interactions; Xu, L., Zein-Sabatto, S. and 
Sekman, A., Development of Intelligent Behaviors for a Mobil Robot. 0-7803-6661-1/01/ 
©2001 IEEE pp. 383-386 - for speaking robot; and BIRON - the Bielefeld Robot 
Companion, http://www.techfak.net/aas/ai/proiects/BIRON/welcome.html - for a robot 
that meets and follows a person. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to LIN B. OLSEN whose telephone number is (571)272- 
9754. The examiner can normally be reached on Mon - Fri, 8:30 -5. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Thomas G. Black can be reached on 571-272-6956. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/L. B. 0.1 

Examiner, Art Unit 3661 
/Thomas G. Black/ 

Supervisory Patent Examiner, Art Unit 3661 



